Information processing apparatus, information processing method, and program

ABSTRACT

The present disclosure relates to an information processing apparatus, an information processing method, and a program that enable to ensure visibility when a virtual object is shielded. An information processing apparatus is provided that includes a display control unit that controls a display so as to display a virtual object by using a first display parameter in a first state where it is determined that the virtual object displayed by the display is hidden by at least one real object as viewed from a first user of the display and display the virtual object by using a second display parameter different from the first display parameter in a second state where it is determined that the virtual object is hidden by real objects more than that in the first state as viewed from the first user. It is possible to apply the present disclosure to, for example, a device included in an augmented reality system.

TECHNICAL FIELD

The present disclosure relates to an information processing apparatus,an information processing method, and a program, and more particularly,to an information processing apparatus, an information processingmethod, and a program that can shield a virtual object and ensurevisibility.

BACKGROUND ART

In recent years, technologies regarding Augmented Reality (AR) thatexpands the real world by superimposing and displaying a virtual objecton a real object in the real world by using various electronicapparatuses have been actively researched and developed.

For example, Patent Document 1 discloses a technology that displays adisplay object (virtual object), related to a real object that can beviewed through a transparent display, on the transparent display havingoptical transparency.

CITATION LIST Patent Document

Patent Document 1: Japanese Patent Application Laid-Open No. 2013-15796

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

By the way, in order to reduce uncomfortable feeling when the virtualobject is superimposed and displayed on the real world and to give aclue regarding a depth to a user, there is a case where shieldingprocessing of shielding the virtual object by the real object isexecuted.

However, by executing such shielding processing, there is a possibilitythat the virtual object is excessively shielded by the real object andcannot be viewed depending on a positional relationship with thesurrounding real object. Therefore, a technology for ensuring visibilityof the virtual object is required.

The present disclosure has been made in view of such a situation, andmakes it possible to shield a virtual object and ensure visibility.

Solutions to Problems

An information processing apparatus according to one aspect of thepresent disclosure is an information processing apparatus including adisplay control unit that controls a display so as to display a virtualobject by using a first display parameter in a first state where it isdetermined that the virtual object displayed by the display is hidden byat least one real object as viewed from a first user of the display anddisplay the virtual object by using a second display parameter differentfrom the first display parameter in a second state where it isdetermined that the virtual object is hidden by real objects more thanthat in the first state as viewed from the first user.

The information processing apparatus according to one aspect of thepresent disclosure may be an independent apparatus or may be an internalblock configuring a single apparatus.

An information processing method and a program according to one aspectof the present disclosure are respectively an information processingmethod and a program corresponding to the information processingapparatus according to the one aspect of the present disclosure.

In the information processing apparatus, the information processingmethod, and the program according to one aspect of the presentdisclosure, in a first state where it is determined that a virtualobject displayed by a display is hidden by at least one real object asviewed from a first user of the display, the virtual object is displayedby using a first display parameter, and in a second state where it isdetermined that the virtual object is hidden by the real objects morethan that in the first state as viewed from the first user, the virtualobject is displayed by using a second display parameter different fromthe first display parameter.

Effects of the Invention

According to one aspect of the present disclosure, it is possible toshield a virtual object and ensure visibility.

Note that the effects described herein are not limited and that theeffect may be any effects described in the present disclosure.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating an example of a configuration of anaugmented reality system to which the technology according to thepresent disclosure is applied.

FIG. 2 is a diagram illustrating an example of use of the augmentedreality system to which the technology according to the presentdisclosure is applied.

FIG. 3 is a block diagram illustrating an example of a configuration ofa client terminal and a position sharing server.

FIG. 4 is a block diagram illustrating an example of a detailedconfiguration of a display method determination unit.

FIG. 5 is a diagram illustrating an example of shielding a virtualobject when the augmented reality system is used by a plurality ofusers.

FIG. 6 is a time-series diagram illustrating a state of a virtual objecthidden by a real object.

FIG. 7 is a diagram illustrating a first example of a display positionof the shielded virtual object.

FIG. 8 is a diagram illustrating a second example of the displayposition of the shielded virtual object.

FIG. 9 is a flowchart for explaining an operation of a client terminal.

FIG. 10 is a flowchart for explaining a flow of first display methoddetermination processing.

FIG. 11 is a diagram illustrating a first example of a transmittance ofa color of the shielded virtual object.

FIG. 12 is a diagram illustrating a second example of the transmittanceof the color of the shielded virtual object.

FIG. 13 is a diagram illustrating an example of arrangement of users andobjects when the transmittance of the color of the shielded virtualobject is changed.

FIG. 14 is a flowchart for explaining a flow of second display methoddetermination processing.

FIG. 15 is a flowchart for explaining a flow of third display methoddetermination processing.

FIG. 16 is a block diagram illustrating an example of a configuration ofthe client terminal and a map providing server.

FIG. 17 is a diagram illustrating an example of use of the augmentedreality system to which the technology according to the presentdisclosure is applied.

FIG. 18 is a diagram schematically illustrating a state of the virtualobject hidden by the real objects.

FIG. 19 is a flowchart for explaining an operation of the clientterminal.

FIG. 20 is a diagram illustrating an example of a configuration of acomputer.

MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of the technology (the present technology)according to the present disclosure will be described with reference tothe drawings. Note that the description will be made in the followingorder.

1. First Embodiment

2. Second Embodiment

3. Modification

4. Configuration of Computer

1. First Embodiment

(Example of System Configuration)

FIG. 1 is a diagram illustrating an example of a configuration of anaugmented reality system to which the technology according to thepresent disclosure is applied.

In FIG. 1, the augmented reality system includes client terminals 10-1to 10-N (N: integer equal to or more than one), a position sharingserver 20, and a network 50. Each of the client terminals 10-1 to 10-Ncan exchange data with the position sharing server 20 via the network 50including the Internet, a mobile phone network, or the like.

The client terminal 10-1 is, for example, an electronic apparatus(information processing apparatus) such as a wearable terminal such as aglass-type information terminal, a Head Mounted Display (HMD), asmartphone, a mobile phone, a tablet-type computer, a personal computer,a game machine, or the like. Note that it is mainly assumed that theclient terminal 10-1 be a mobile terminal that can be carried and usedby a user. However, the client terminal 10-1 is not limited to that andmay be a fixed terminal.

For example, the client terminal 10-1 is mounted on a user's head,displays (transmits) a real object that exists in the real world on adisplay device (display) fixed to a direction of a line-of-sight of theuser, and superimposes and displays a virtual object that does not existin the real world (real space) by the Augmented Reality (AR) technology.

Similarly to the client terminal 10-1, each of the client terminals 10-2to 10-N includes an electronic apparatus such as a wearable computer, ahead mounted display, a smartphone, or the like and presents the virtualobject together with the real object to the user.

The position sharing server 20 is a server used to share positioninformation and direction information of each of the client terminals10-1 to 10-N (user who wears the client terminal). The position sharingserver 20 receives the position information and the directioninformation transmitted from the client terminals 10-1 to 10-N via thenetwork 50. Furthermore, the position sharing server 20 transmits theposition information and the direction information received from eachclient terminal 10 to the other client terminal 10 via the network 50.

In the augmented reality system configured as described above, in a casewhere a plurality of users wearing (or holding) the client terminal 10uses the same AR service, when the virtual object is displayed in thereal world, the virtual object is displayed according to the position(viewpoint) of each user. Note that, in the following description, whilea real object (including person such as user) existing in the real world(real space) is referred to as a real object O_(R), a virtual objectthat does not exist in the real world (real space) is referred to as avirtual object O_(V).

Here, FIG. 2 illustrates an example of use of the augmented realitysystem to which the technology according to the present disclosure isapplied. [0026]

In FIG. 2, a case is schematically illustrated where a display object 12is displayed in front of four users when the four users including users11-1 to 11-4 respectively wear the client terminals 10-1 to 10-4 and usethe same AR service. In this case, it can be said that the users 11-1 to11-4 are the real objects O_(R), and the display object 12 is thevirtual object O_(V).

Note that, in the following description, a user to be focused among theplurality of users who receives the provision of the same AR service(for example, AR game) is simply referred to as a user (first user), andusers different from the user to be focused is referred to as the otheruser (second user). The users are distinguished in this way.

For example, for the user 11-1 (first user) wearing the client terminal10-1, the users 11-2 to 11-4 are the other users (second user), or forexample, for the user 11-4 (first user) wearing the client terminal10-4, the users 11-1 to 11-3 can be said as the other users 11 (seconduser).

Each of the client terminals 10-1 to 10-4 can share the positionsthereof in the same coordinates by exchanging the position informationwith the position sharing server 20 via the network 50. Furthermore,here, in addition to the position information, the direction informationindicating a head orientation of each user 11 can be shared.

Specifically, a position P1 and a head orientation D1 of the user 11-1wearing the client terminal 10-1, a position P2 and a head orientationD2 of the user 11-2 wearing the client terminal 10-2, a position P3 anda head orientation D3 of the user 11-3 wearing the client terminal 10-3,and a position P4 and a head orientation D4 of the user 11-4 wearing theclient terminal 10-4 are shared by the client terminals 10-1 to 10-4.

Each of the client terminals 10-1 to 10-4 determines a degree ofshielding the other user 11 with respect to the display object 12 on thebasis of the information such as the position information and thedirection information shared via the position sharing server 20,determines a display parameter corresponding to the determinationresult, and displays the display object 12 by using the determineddisplay parameter.

This display parameter is associated with, for example, an attributesuch as a display position of the display object 12.

In this way, in the augmented reality system to which the technologyaccording to the present disclosure is applied, when each clientterminal 10 displays the virtual object O_(V), a shielding degree of thereal object O_(R) with respect to the virtual object O_(V) isdetermined, and the virtual object O_(V) is displayed on the basis ofthe display parameter corresponding the determination result.

Note that, in FIG. 1, for convenience of description, minimum componentsare illustrated as the augmented reality system. However, othercomponents may be added, for example, to install a dedicated AR serverthat provides functions regarding the AR technology. Furthermore, thereal object O_(R) is not limited to the user 11 and includes realobjects, for example, furniture and buildings.

(Configuration of Client Terminal)

FIG. 3 illustrates an example of a configuration of the client terminal10-1 in FIG. 1.

In FIG. 3, the client terminal 10-1 includes a measurement unit 101, aposition estimation unit 102, a communication unit 103, a display methoddetermination unit 104, a display control unit 105, and a display unit106.

The measurement unit 101 includes various sensors, for example, a stereocamera, a gyro sensor, an acceleration sensor, or the like. Themeasurement unit 101 performs various measurements according to anoperation of the user 11-1 wearing the client terminal 10-1 and suppliesmeasured information obtained as a result of the measurement to theposition estimation unit 102.

The measured information includes, for example, information such assensor information including angular speed information measured by thegyro sensor, acceleration information measured by the accelerationsensor, or the like in addition to distance information measured by thestereo camera.

The position estimation unit 102 estimates a position of the clientterminal 10-1 on the basis of the measured information supplied from themeasurement unit 101 and supplies the position information obtained as aresult of the estimation to the communication unit 103 and the displaymethod determination unit 104. Here, by using information such as thedistance information as the measured information, for example, it ispossible to estimate the position of the user 11-1 with respect to areal object around the user 11-1 (relative position).

Furthermore, the position estimation unit 102 estimates the headorientation of the user 11-1 wearing the client terminal 10-1 on thebasis of the measured information supplied from the measurement unit 101and supplies the direction information obtained as a result of theestimation to the communication unit 103 and the display methoddetermination unit 104. Here, by using information such as the angularspeed information, the acceleration information, or the like as themeasured information, for example, the orientation (posture) of the headof the user 11-1 can be estimated.

The communication unit 103 includes, for example, a communication moduleor the like compliant to wireless communication such as a wireless LocalArea Network (LAN), cellular communication (for example, LTE-Advanced,5G, or the like), or the Bluetooth (registered trademark) or wiredcommunication such as the Ethernet (registered trademark).

The communication unit 103 transmits the position information and thedirection information of the user 11-1 supplied from the positionestimation unit 102 to the position sharing server 20 via the network50. Furthermore, the communication unit 103 receives the positioninformation and the direction information of the other users 11-2 to11-N transmitted from the position sharing server 20 via the network 50and supplies the received information to the display methoddetermination unit 104.

The position information and the direction information of the user 11-1from the position estimation unit 102 and the position information andthe direction information of the other users 11-2 to 11-N from thecommunication unit 103 are supplied to the display method determinationunit 104. The display method determination unit 104 determines a methodof displaying the virtual object O_(V) on the basis of the positioninformation and the direction information of the user 11-1 and theposition information and the direction information of the other users11-2 to 11-N.

Here, FIG. 4 illustrates a detailed configuration of the display methoddetermination unit 104. In FIG. 4, the display method determination unit104 includes a position information acquisition unit 111, a shieldingdetermination unit 112, a shielding rate calculation unit 113, a displayparameter setting unit 114, and an attribute determination unit 115.

The position information acquisition unit 111 acquires the positioninformation and the direction information of the user 11-1 estimated bythe position estimation unit 102 or the position information and thedirection information of the other users 11-2 to 11-N received by thecommunication unit 103 and supplies the acquired information to theshielding determination unit 112, the shielding rate calculation unit113, and the display parameter setting unit 114.

The shielding determination unit 112 determines whether or not thevirtual object O_(V) is hidden by the real object O_(R) on the basis ofinformation such as the position information and the directioninformation supplied from the position information acquisition unit 111and supplies the determination result to the shielding rate calculationunit 113 and the display parameter setting unit 114. Here, the degree ofshielding of the real object O_(R) with respect to the virtual objectO_(V) is determined. However, the degree of shielding is not limited tothe degree of shielding when the object is actually hidden, and mayinclude the degree of shielding when the object may be hidden.

When the determination result supplied from the shielding determinationunit 112 indicates that the shielding is performed, the shielding ratecalculation unit 113 calculates a shielding rate (shielding degree) ofthe real object O_(R) with respect to the virtual object O_(V) on thebasis of the information such as the position information and thedirection information supplied from the position information acquisitionunit 111 and supplies the calculation result to the display parametersetting unit 114.

The display parameter setting unit 114 generates a display parameter onthe basis of the information such as the position information and thedirection information supplied from the position information acquisitionunit 111, the determination result regarding shielding supplied from theshielding determination unit 112, and the calculation result of theshielding rate from the shielding rate calculation unit 113 and supplies(set) the display parameter to the display control unit 105.

The attribute determination unit 115 confirms an attribute of thevirtual object O_(V), determines whether or not the virtual object O_(V)is associated with a specific position in the real world, and suppliesthe determination result to the display parameter setting unit 114. Atthis time, when generating the display parameter, the display parametersetting unit 114 uses the determination result supplied from theattribute determination unit 115.

Returning to FIG. 3, the display control unit 105 performs control todisplay information regarding the virtual object O_(V) or the like onthe display unit 106 on the basis of the display parameter supplied fromthe display method determination unit 104. Note that the display controlunit 105 may generate the display parameter by including the positioninformation acquisition unit 111 to the attribute determination unit 115(FIG. 4).

The display unit 106 includes, for example, a display device (display)such as a Liquid Crystal Display (LCD), an Organic Light Emitting Diode(OLED), a projector, or the like. The display unit 106 displays theinformation regarding the virtual object O_(V) or the like (video orimage including the same) according to the control from the displaycontrol unit 105.

Note that, in the client terminal 10-1, the position estimation unit102, the display method determination unit 104, or the display controlunit 105 may be implemented by executing a program by a processor (CPU1001 in FIG. 20) such as a Central Processing Unit (CPU) or may beimplemented by dedicated hardware.

Furthermore, in FIG. 3, minimum components of the client terminal 10-1are illustrated. However, for example, other components such as a voiceoutput device (for example, speaker, earphone, or the like) used tooutput voice regarding the virtual object O_(V) may be added.

(Configuration of Position Sharing Server)

Furthermore, FIG. 3 illustrates an example of a configuration of theposition sharing server 20 in FIG. 1.

In FIG. 3, the position sharing server 20 includes a control unit 201and a communication unit 202.

The control unit 201 includes, for example, a processor such as a CPUand functions as a central processing apparatus that controls anoperation of each unit and executes various arithmetic processing.

The communication unit 202 includes a communication module or the likecompliant to wireless communication or wired communication.

The communication unit 202 receives the position information and thedirection information transmitted from the client terminal 10-1 via thenetwork 50 according to the control from the control unit 201 andtransmits the received information to each of the client terminals 10-2to 10-N via the network 50.

Furthermore, the communication unit 202 receives the positioninformation and the direction information transmitted from the clientterminals 10-2 to 10-N via the network 50 according to the control fromthe control unit 201 and transmits the received information to theclient terminal 10-1 via the network 50.

Note that, because the client terminals 10-2 to 10-N are similarlyconfigured to the client terminal 10-1 in FIG. 3, detailed descriptionhere is omitted.

(Problems and Solutions When Virtual Object O_(V) Is Shielded) VariousAR services can be provided by using the augmented reality system towhich the technology according to the present disclosure is applied.However, here, as an AR service, for example, an AR game that is a game,using augmented reality (AR), played by a plurality of users 11 incooperation with each other will be described as an example.

For example, in the AR game, there is a case where the virtual objectO_(V) is superimposed and displayed on the real world in accordance witha progress of cooperative play by the plurality of users 11-1 to 11-Nrespectively wearing the client terminals 10-1 to 10-N.

In such a case, in order to reduce uncomfortable feeling when thevirtual object O_(V) is superimposed and displayed on the real world orgive a clue regarding the depth to the user 11, there is a case whereshielding processing of shielding the virtual object O_(V) (for example,display object 12 such as character, item, or the like of AR game) bythe real object O_(R) (for example, other user 11 or the like) isexecuted.

By executing such shielding processing, while it is possible to reducethe uncomfortable feeling and give a clue regarding the depth, thevirtual object O_(V) is shielded by the real object O_(R). Therefore,there is a possibility that the user 11 cannot see the virtual objectO_(V) according to the shielding situation.

Here, in FIG. 5, a scene in which the plurality of users 11cooperatively plays the AR game is illustrated. Note that FIG. 5 is afield of view of the user 11-4 positioned behind the users 11-1 to 11-3when the four users including the users 11-1 to 11-4 use the same ARservice as respectively wearing the client terminals 10-1 to 10-4.

In FIG. 5, in the middle of the progress of the AR game, the displayobject 12 such as characters, items, or the like is displayed. However,most of the display object 12 is shielded by the users 11-2 and 11-3.Therefore, from the user 11-4 positioned behind the users 11-2 and 11-3,the display object 12 is shielded by the users 11-2 and 11-3, and theuser 11-4 can only visually recognize a part of the display object 12(small part on lower left side in FIG. 5).

In this way, according to the current technology, because there is apossibility that the virtual object O_(V) is excessively shielded andcannot be visually recognized depending on the situation (positionalrelationship with surrounding real object O_(R)), it is required toensure visibility when the virtual object O_(V) is shielded by the realobject O_(R). Therefore, regarding the augmented reality system to whichthe technology according to the present disclosure is applied, atechnology of achieving shielding of the virtual object O_(V) andensuring the visibility when the virtual object O_(V) is shielded by thereal object O_(R) is proposed.

Note that the shielding processing according to the technology of thepresent disclosure may be assumed as processing of displaying thevirtual object O_(V) as if the virtual object O_(V) was hidden by thereal object O_(R). More specifically, the shielding processing may beassumed as processing of not displaying a part of the virtual objectO_(V) on the basis of a shape of the real object O_(R) disposed in frontof the virtual object O_(V) as viewed from the user. In other words, asa result of such non-display processing, a part of the virtual objectO_(V) that overlaps with the real object O_(R) is cut. Such displayprocessing of the virtual object O_(V) may be referred to as “occlusion”in the field of the augmented reality technology.

FIG. 6 illustrates a state of the display object 12 that may be shieldedby the other user 11 in time series. Note that, A of FIG. 6schematically illustrates a field of view of each user 11 at a certaintime t11, and B of FIG. 6 schematically illustrates a field of view ofeach user 11 at a time t12 subsequent to the time t11.

In A of FIG. 6, fields of view V1 to V4 of the respective users 11-1 to11-4 contain most of the display object 12 and are not shielded by theother user 11. Therefore, at the time t11, the users 11-1 to 11-4 canvisually recognize the display object 12.

On the other hand, in B of FIG. 6, although the fields of view V1 to V4of the respective users 11-1 to 11-4 contain most or a part of thedisplay object 12, a part of the field of view of the user 11 isshielded by a part of the other user 11. More specifically, for example,the field of view V4 of the user 11-4 contains the user 11-2 who existsin front of the user 11-4. Therefore, at the time t12, not all the users11-1 to 11-4 cannot visually recognize the display object 12.

In this way, when the number of real objects O_(R) that may hide thevirtual object O_(V) is large (for example, four users), even if a partof the virtual object O_(V) that can be seen at a certain time (forexample, time t11) is large, there is a high possibility that thevirtual object O_(V) is hidden in the future (for example, time t12).

Therefore, in the augmented reality system to which the technologyaccording to the present disclosure is applied, on the basis of thenumber of real objects O_(R) hiding the virtual object O_(V) and adegree of shielding such as the shielding rate (shielding rate), thevirtual object O_(V) is displayed at (moved to) a position where all theusers 11-1 to 11-4 can see the virtual object O_(V).

FIG. 7 illustrates a first example of a display position of the virtualobject O_(V) shielded by the real object O_(R). In FIG. 7, although atiming when the display object 12 is displayed comes in the middle ofthe progress of the AR game, if the display object 12 is displayed at adefault position, the display object 12 is shielded by two users (otherusers 11-1 and 11-2) from the user 11-4 as indicated by a dotted framein FIG. 7.

Therefore, in FIG. 7, the display object 12 is moved to a position abovethe default position by a predetermined distance as indicated by anarrow in FIG. 7 and is displayed at the position where all the users11-1 to 11-4 can visually recognize the display object 12. With thisoperation, when the virtual object O_(V) is displayed, as in a casewhere the shielding processing is executed, it is possible to reduce theuncomfortable feeling and give a clue regarding the depth by the realobject O_(R) (other users 11-1 to 11-3), and it is possible to ensurethe visibility of the virtual object O_(V).

Furthermore, FIG. 8 illustrates a second example of the display positionof the virtual object O_(V) shielded by the real object O_(R). In FIG.8, when the display object 12 is displayed at the default position at atiming of displaying the display object 12, a part of the display object12 (small part on lower left side in FIG. 8) is shielded by the singleuser (other user 11-1) from the user 11-4.

In this case, even if the display object 12 is displayed at the defaultposition, the users 11-1 to 11-3 can visually recognize the entiredisplay object 12. Although it is not possible for the user 11-4 tovisually recognize the small part on the lower left side of the displayobject 12, the user 11-4 can visually recognize most of the displayobject 12. Therefore, it is possible to recognize content, a state, orthe like of the display object 12.

Therefore, in FIG. 8, the display object 12 is displayed at the defaultposition and is displayed at the position where all the users 11-1 to11-4 can visually recognize the display object 12. With this operation,when the virtual object O_(V) is displayed, it is possible to reduce theuncomfortable feeling and give a clue regarding the depth by the realobject O_(R), and it is possible to ensure the visibility of the virtualobject O_(V).

Note that, in FIG. 8, it has been described that the display object 12is displayed at the default position. However, the display object 12 maybe moved from the default position to the right by a predetermineddistance and be displayed at the position where all the users 11including the user 11-4 can visually recognize the complete displayobject 12.

Here, when the state illustrated in the second example in FIG. 8 iscompared with the state illustrated in the first example in FIG. 7, inthe state (first state) illustrated in the second example in FIG. 8, itis determined that the virtual object O_(V) (display object 12) ishidden by at least one real object O_(R) (one user: other user 11-1),and the virtual object O_(V) (display object 12) is displayed by usingthe display parameter (first display parameter) associated with thedisplay position (default position or position away from defaultposition to right side by predetermined distance) of the virtual objectO_(V) (display object 12).

On the other hand, in the state (second state) illustrated in the firstexample in FIG. 7, it is determined that the virtual object O_(V)(display object 12) is hidden by the real objects O_(R) (two users:other users 11-1 and 11-2) more than that in the state (first state)illustrated in the second example in FIG. 8, and the virtual objectO_(V) (display object 12) is displayed by using the display parameter(second display parameter) associated with the display position(position away from default position to upper side by predetermineddistance) of the virtual object O_(V) (display object 12).

(Operation of Client Terminal)

Next, an operation of the client terminal 10 described above will bedescribed with reference to the flowcharts in FIGS. 9 and 10. Note that,here, as in the examples in FIGS. 5 to 8, an example of an operation ofthe client terminal 10-4 worn by the user 11-4 among the users 11-1 to11-4 who are playing the AR game in cooperation with each other will bedescribed.

In the client terminal 10-4, a map is loaded (S11). That is, bygenerating three-dimensional map information (map) regardingsurroundings of the images in advance on the basis of the plurality ofcaptured images captured by the measurement unit 101 (camera thereof) inadvance, the client terminal 10-4 can load a map (that is, internallygenerated map) corresponding to the current position.

In step S12, the position information acquisition unit 111 acquires theposition information and the direction information of the user 11-4.

In step S13, the shielding determination unit 112 determines whether ornot the virtual object O_(V) such as the display object 12 is hidden bythe real object O_(R) (for example, furniture, buildings, or the like onmap) other than the other users 11-1 to 11-3 on the basis of thesituation such as the position, the direction, or the like of the user11-4 on the loaded map.

When the determination processing in step S13 is affirmative (“YES” inS13), the procedure proceeds to step S14. In step S14, the shieldingrate calculation unit 113 calculates a shielding rate of the real objectO_(R) that shields the virtual object O_(V).

When the processing in step S14 ends, the procedure proceeds to stepS15. Furthermore, when the determination processing in step S13 isnegative (“NO” in S13), step S14 is skipped, and the procedure proceedsto step S15.

In step S15, the position information acquisition unit 111 acquires theposition information and the direction information of the other users11-1 to 11-3.

In step S16, the shielding determination unit 112 determines whether ornot the virtual object O_(V) such as the display object 12 is hidden bythe other users 11-1 to 11-3 on the basis of the situation such theposition, the direction, or the like of the users 11-1 to 11-3 on theloaded map.

When the determination processing in step S16 is affirmative (“YES” inS16), the procedure proceeds to step S17. In step S17, the displaymethod determination unit 104 executes display method determinationprocessing.

In this display method determination processing, a degree of shieldingof the real object O_(R) with respect to the virtual object O_(V) isdetermined, and a display parameter corresponding to the determinationresult is set.

Here, the degree of shielding includes, for example, the number of otherusers 11-1 to 11-3 that exist between the display object 12 and the user11-4 and hide the display object 12, a rate of a region of the displayobject 12 hidden by the other users 11-1 to 11-3 with respect to theentire display object 12 (shielding rate), or the like. Furthermore,here, the display parameter is associated with an attribute that is adisplay position of the display object 12.

Note that details of the display method determination processing will bedescribed later with reference to the flowchart in FIG. 10.

When the processing in step S17 ends, the procedure proceeds to stepS18. Furthermore, when the determination processing in step S16 isnegative (“NO” in S16), step S17 is skipped, and the procedure proceedsto step S18.

In step S18, the display control unit 105 controls display of thevirtual object O_(V).

For example, in this display control processing, when the determinationprocessing in step S13 is affirmative (“YES” in S13), the virtual objectO_(V) (display object 12) is displayed while being shielded by the realobject O_(R) (for example, real object O_(R) such as furniture,buildings, or the like, other than other users 11-1 to 11-3) at apredetermined rate on the basis of the shielding rate calculated in theprocessing in step S14.

Furthermore, for example, in this display control processing, when thedetermination processing in step S16 is affirmative (“YES” in S16), thevirtual object O_(V) (display object 12) is displayed on the basis ofthe display parameter set in the processing in step S17.

When the processing in step S18 ends, the procedure proceeds to stepS19. In step S19, it is determined whether or not to end the processing.

When the determination processing in step S19 is negative (“NO” in S19),the procedure returns to step S12, and the processing in steps S12 toS19 is repeated, and the display of the virtual object O_(V) withrespect to the real object O_(R) is controlled. Furthermore, when thedetermination processing in step S19 is affirmative (“YES” in S19), theprocessing illustrated in FIG. 9 ends.

The operation of the client terminal 10 has been described above.

Note that, in the client terminal 10, a timing when the display of thevirtual object O_(V) is controlled is, for example, a timing when thevirtual object O_(V), which has been already displayed, is shielded bythe real object O_(R) in addition to a timing, for example, when thevirtual object O_(V) is superimposed and displayed on the real world.

(Flow of First Display Method Determination Processing)

Next, a flow of first display method determination processingcorresponding to step S17 in FIG. 9 will be described with reference tothe flowchart in FIG. 10.

In step S111, the shielding determination unit 112 determines whether ornot the number of the other users 11 hiding the virtual object O_(V)such as the display object 12 and the shielding rate are less thanthresholds.

When the determination processing in step S111 is affirmative (“YES” inS111), the procedure proceeds to step S112. In step S112, the displayparameter setting unit 114 sets a first display parameter used todisplay the virtual object O_(V) at a first position where all the userscan visually recognize the virtual object O_(V).

With this setting, for example, when the display object 12 is shieldedby the single user (other user 11-1) and the shielding rate is less thana predetermined rate as illustrated in FIG. 8, the first displayparameter used to display the display object 12 at the default positionis set (S112 in FIG. 10), and the display object 12 is displayed at thefirst position where all the users 11-1 to 11-4 can be visuallyrecognize the display object 12 (S18 in FIG. 9).

When the determination processing in step S111 is negative (“NO” inS111), the procedure proceeds to step S113. In step S113, the displayparameter setting unit 114 sets a second display parameter used todisplay the virtual object O_(V) at a second position where all theusers can visually recognize virtual object O_(V).

With this setting, for example, when the display object 12 is shieldedby the two users (other users 11-1 and 11-2) and the shielding rate isequal to or more than a predetermined rate as illustrated in FIG. 7, thesecond display parameter used to move the display object 12 to thesecond position above the default position (change display position) isset (S113 in FIG. 10), and the display object 12 is displayed at thesecond position where all the users 11-1 to 11-4 can visually recognizethe display object 12 (S18 in FIG. 9).

When the processing in step S112 or S113 ends, the procedure returns tostep S17 in FIG. 9, and the processing in step S17 and subsequent stepsis executed.

In this way, in the first display method determination processing, whenthe degree of shielding is determined, in the first state (stateillustrated in second example in FIG. 8) where it is determined that thevirtual object O_(V) (display object 12) is hidden by at least a singlereal object O_(R) (one user: other user 11-1), the virtual object O_(V)(display object 12) is displayed by using the first display parameter(parameter used for display at first display position).

On the other hand, in the first display method determination processing,when the degree of shielding is determined, in the second state (stateillustrated in first example in FIG. 7) where it is determined that thevirtual object O_(V) (display object 12) is hidden by the real objectsO_(R) (two users: other users 11-1 and 11-2) more than that in the firststate, the virtual object O_(V) (display object 12) is displayed byusing the second display parameter (parameter used for display at seconddisplay position).

Note that, in the first display method determination processing, whenthe degree of shielding is determined in the determination processing instep S111, the number of other users hiding the virtual object O_(V) andthe shielding rate are included in determination targets. However, it ispossible that the shielding rate is not included in the determinationtarget. Furthermore, if the number of other users 11 hiding the virtualobject O_(V) is included in the determination target regarding thedegree of shielding, another index representing the degree of shieldingmay be used. Furthermore, the threshold to be a determination criterioncan be appropriately adjusted, for example, according to content, ascene, or the like of the AR service.

As described above, in the first embodiment, the display parameterassociated with the display position of the virtual object O_(V) ischanged according to the number of real objects O_(R) shielding thevirtual object O_(V) and the virtual object O_(V) is displayed at theposition where all the users can visibility recognize the virtual objectO_(V) so that the visibility at the time when the virtual object O_(V)is shielded is ensured. In particular, when the client terminal 10 is amobile terminal, a display region of a display device (display) isnarrow. However, by changing the display position of the virtual objectO_(V) according to the number of real objects O_(R) shielding thevirtual object O_(V), the visibility of the virtual object O_(V) can beensured.

2. Second Embodiment

By the way, some virtual objects O_(V) are associated with a specificposition in the real world, and some virtual objects O_(V) are notassociated with the specific position in the real world. Here, if thevirtual object O_(V) is moved to a position where the virtual objectO_(V) can be visually recognized when the virtual object O_(V) isassociated with the real world, the movement may be unnatural.

Therefore, the augmented reality system to which the technologyaccording to the present disclosure is applied has the followingconfiguration in order to enable visibility of the virtual object O_(V)at the time of being shielded by a real object O_(R) to be ensured whenthe virtual object O_(V) is associated with the real world.

That is, in the augmented reality system to which the technologyaccording to the present disclosure is applied, a transmittance of acolor of the virtual object O_(V) is changed on the basis of a degree ofshielding such as the number of real objects O_(R) hiding the virtualobject O_(V) or a shielded rate (shielding rate).

FIG. 11 illustrates a first example of the transmittance of the color ofthe virtual object O_(V) shielded by the real object O_(R). In FIG. 11,although a timing when a display object 12 is displayed comes in themiddle of progress of an AR game, if the display object 12 is displayedwith a default color, most of the display object 12 is shielded by asingle user (other user 11-1) from a user 11-4.

Therefore, in FIG. 11, the transmittance of the color of the displayobject 12 is changed, and the display object 12 (part thereof) isoverlapped with the other user 11-1 (part thereof) and is displayed.With this operation, when the virtual object O_(V) is displayed, as in acase where the shielding processing is executed, it is possible toreduce the uncomfortable feeling and give a clue regarding the depth bythe real object O_(R) (other users 11-1 to 11-3), and it is possible toensure the visibility of the virtual object O_(V).

Furthermore, FIG. 12 illustrates a second example of the transmittanceof the color of the virtual object O_(V) shielded by the real objectO_(R). In FIG. 12, if the display object 12 is displayed with a defaultcolor at a display timing of the display object 12, most of the displayobject 12 is shielded by two users (other users 11-2 and 11-3) from theuser 11-4.

Therefore, in FIG. 12, the transmittance of the color of the displayobject 12 is changed, and the display object 12 (part thereof) isoverlapped with the other users 11-2 and 11-3 (part thereof) and isdisplayed. With this operation, when the virtual object O_(V) isdisplayed, it is possible to reduce the uncomfortable feeling and give aclue regarding the depth by the real object O_(R), and it is possible toensure the visibility of the virtual object O_(V).

However, the transmittance (second transmittance) of the color of thedisplay object 12 superimposed on the two users (other users 11-2 and11-3) in FIG. 12 is higher than the transmittance (first transmittance)of the color of the display object 12 superimposed on the single user(other user 11-1) in FIG. 11, and the back of the two users (other users11-2 and 11-3) shielding the display object 12 can be relatively clearlyseen. In other words, the first transmittance (FIG. 11) is lower thanthe second transmittance (FIG. 12), and it can be said that the back ofthe single user (other user 11-1) shielding the display object 12 isslightly blurredly seen.

More specifically, the first transmittance (FIG. 11) and the secondtransmittance (FIG. 12) are higher than zero % and lower than 100% whenit is assumed that complete transmission be 100% and complete opacity iszero %. Furthermore, in this case, the first transmittance (for example,transmittance 40%) is lower than the second transmittance (for example,transmittance 60%).

Furthermore, at this time, for example, when a real object 13 (forexample, real object O_(R) fixed to real space such as furniture,buildings, or the like) other than the other users 11-1 to 11-3 existsas the real object O_(R) as illustrated in FIG. 13, it can be determinedwhether or not the display object 12 is shielded by the real objectO_(R) depending on whether or not the real object O_(R) is the otherusers 11-1 to 11-3 or the real object 13. Note that, in the presentdisclosure, there is a case where the real object 13 is referred to as astatic real object 13.

For example, when a display object 12-1 exists between the other users11-1 to 11-3 and the static real object 13, the display object 12-1 isnot completely shielded by the other users 11-1 to 11-3, and the displayobject 12-1 is displayed at the transmittance of the color according tothe degree of shielding (the number of users 11 shielding the displayobject 12-1 or the like).

On the other hand, for example, when a display object 12-2 exists behindthe static real object 13, the static real object 13 completely shieldsthe display object 12-2 (set transmittance to zero %), and a part of thedisplay object 12-2 can be seen or the entire display object 12-2 cannotbe seen from the users 11-1 to 11-4 according to the shielding rate.

In this way, in the AR game, even if the virtual object O_(V) is behindthe other user 11 who performs cooperative play, the virtual objectO_(V) is not completely shielded. However, when the virtual object O_(V)is shielded by the static real object 13 other than the other user 11who performs cooperative play, the virtual object O_(V) is shielded, andthe shielded part cannot be completely seen from each user 11. In otherwords, while the virtual object O_(V) is completely shielded by thestatic real object 13, for example, furniture, buildings, or the like, adisplay state of the virtual object O_(V) is changed and the virtualobject O_(V) is not completely shielded by the user 11 that is a dynamicobject.

More specifically, it is determined whether the real object O_(R) in thereal space is a static object or a dynamic object. Here, the real objectO_(R) that is determined as a dynamic object is continuously tracked bythe augmented reality system to which the technology according to thepresent disclosure is applied, and processing of shielding the virtualobject O_(V) by the dynamic object is prevented or inhibited. As aresult, the shielding processing can be simplified, and a processingload of a processor or the like can be lowered. Note that the preventionand the inhibition of the shielding processing in the technologyaccording to the present disclosure will be described later in detail.

(Flow of Second Display Method Determination Processing)

Next, an operation of a client terminal 10 will be described. In thesecond embodiment, although the processing of the flowchart illustratedin FIG. 9 is executed as in the first embodiment, processing content ofthe display method determination processing in step S17 in FIG. 9 isdifferent.

Therefore, here, a flow of the second display method determinationprocessing corresponding to step S17 in FIG. 9 will be described withreference to the flowchart in FIG. 14. Furthermore, here, a displayparameter is associated with an attribute that is the transmittance ofthe display object 12.

In step S121, a shielding determination unit 112 determines whether ornot the number of the other users 11 hiding the virtual object O_(V)such as the display object 12 is less than a threshold.

When the determination processing in step S121 is affirmative (“YES” inS121), the procedure proceeds to step S122. In step S122, a displayparameter setting unit 114 sets a first display parameter used todisplay the color of the virtual object O_(V) at the firsttransmittance.

With this setting, for example, when the display object 12 is shieldedby the single other user 11-1 and the number is less than a threshold asillustrated in FIG. 11, the first display parameter used to display thecolor of the display object 12 at the first transmittance (for example,transmittance 40%) lower than the second transmittance (for example,transmittance 60%) is set, and the display object 12 is displayed at thedefault position so that the transmittance of the color of a partshielded by the other user 11-1 is set to the first transmittance (forexample, transmittance 40%).

When the determination processing in step S121 is negative (“NO” inS121), the procedure proceeds to step S123. In step S123, the displayparameter setting unit 114 sets a second display parameter used todisplay the color of the virtual object O_(V) at the secondtransmittance.

With this setting, for example, when the display object 12 is shieldedby the two other users 11-2 and 11-3 and the number is equal to or morethan the threshold as illustrated in FIG. 12, the second parameter usedto display the color of the display object 12 at the secondtransmittance (for example transmittance 60%) higher than the firsttransmittance (for example, transmittance 40%) is set, and the displayobject 12 is displayed at the default position so that the transmittanceof the color of a part shielded by the other users 11-2 and 11-3 is setto the second transmittance (for example, transmittance 60%).

When the processing in step S122 or S123 ends, the procedure returns tostep S17 in FIG. 9, and the processing in step S17 and subsequent stepsis executed.

In this way, in the second display method determination processing, whena degree of shielding is determined, the number of real objects O_(R)(other user 11) hiding the virtual object O_(V) (display object 12) isused for determination and is compared with the threshold to be adetermination criterion. While the first display parameter (firsttransmittance) is set when the number of other users 11 is less than thethreshold, the second display parameter (second transmittance) is setwhen the number of other users 11 is equal to or more than thethreshold.

Note that the threshold to be the determination criterion can beappropriately adjusted, for example, according to content, a scene, orthe like of an AR service. Furthermore, here, one threshold is used asthe determination criterion. However, for example, when displayparameters according to three or more transmittances can be set, forexample, a plurality of thresholds such as a first threshold, a secondthreshold, or the like may be used as determination criteria.

Moreover, here, a case where the number of other users 11 hiding thevirtual object O_(V) (display object 12) is used for a determinationtarget of the degree of shielding has been described as an example.However, as long as the determination target includes the number ofother users 11 hiding the virtual object O_(V), the determination targetmay further include, for example, another index indicating a degree ofshielding such as a shielding rate.

By the way, the first display method determination processing describedin the first embodiment and the second display method determinationprocessing described in the second embodiment may be selectivelyexecuted, and a flow of such processing will be described with referenceto the flowchart in FIG. 15.

(Flow of Third Display Method Determination Processing)

FIG. 15 is a flowchart for explaining a flow of third display methoddetermination processing corresponding to step S17 in FIG. 9.

In step S131, an attribute determination unit 115 confirms an attributeof the virtual object O_(V) to be displayed.

In step S132, the attribute determination unit 115 determines whether ornot the virtual object O_(V) is associated with a specific position inthe real world on the basis of the confirmation result of the processingin step S131.

When the determination processing in step S132 is affirmative (“YES” inS132), the procedure proceeds to step S133. In step S133, the displaymethod determination unit 104 executes the second display methoddetermination processing (FIG. 14).

In the second display method determination processing, on the premisethat the virtual object O_(V) (display object 12) is an object (forexample, plant, building, or the like displayed when AR game is played)fixed to a predetermined surface in the real world (real space) and isdisplayed at a default position, a display parameter used to display thevirtual object O_(V) at a transmittance of a color corresponding to thedegree of shielding is set.

When the determination processing in step S132 is negative (“NO” inS132), the procedure proceeds to step S134. In step S134, the displaymethod determination unit 104 executes the first display methoddetermination processing (FIG. 10).

In the first display method determination processing, on the premisethat the virtual object O_(V) (display object 12) is an object (forexample, message, symbol, or the like displayed when AR game is played)that is not fixed to a surface in the real world (real space) and can bedisplayed while being moved from the default position, the displayparameter used to display the virtual object O_(V) at the positionaccording to the degree of shielding is set.

When the processing in step S133 or S134 ends, the procedure returns tostep S17 in FIG. 9, and the processing in step S17 and subsequent stepsis executed.

In this way, in the third display method determination processing,because the display method can be switched according to whether or notthe virtual object O_(V) is associated with the specific position in thereal world, that is, whether or not the virtual object O_(V) is theobject fixed to the predetermined surface in the real world or whetheror not the virtual object O_(V) is an object that is not fixed to thesurface in the real world, the virtual object O_(V) can be moreappropriately displayed, for example, according to the content, thescene, or the like of the AR service.

As described above, in the second embodiment, the display parameterassociated with the transmittance of the virtual object O_(V) (colorthereof) is changed according to the number of real objects O_(R)shielding the virtual object O_(V) and the virtual object O_(V) isdisplayed at the default position so that the visibility is ensured whenthe virtual object O_(V) is shielded. In particular, when the clientterminal 10 is a mobile terminal, a display region of a display device(display) is narrow. However, by changing the transmittance of thevirtual object O_(V) according to the number of real objects O_(R)shielding the virtual object O_(V), the visibility of the virtual objectO_(V) can be ensured.

Note that, in the first embodiment, a case where the display parameteris associated with the attribute that is the display position of thevirtual object O_(V) has been described, and in the second embodiment, acase where the display parameter is associated with the attribute thatis the transmittance of the virtual object O_(V) has been described.However, the display parameter may be associated with the attributeother than the display position and the transmittance.

That is, the display parameter can be associated with at least one ofattributes, for example, the display position of the virtual objectO_(V), the transmittance of the virtual object O_(V) with respect to thereal object O_(R), a display posture of the virtual object O_(V), a sizeof the virtual object O_(V), and the color of the virtual object O_(V).For example, when the display parameter is associated with the attributesuch as the display posture or the size, the display posture or the sizeof the display object 12 is adjusted and displayed according to thedegree of shielding such as the number of other users 11 hiding thedisplay object 12.

In the first and second embodiments, the first state and the secondstate are determined after the virtual object O_(V) is actually hiddenby the real object O_(R). Instead of these determinations, the firststate and the second state may be determined before the virtual objectO_(V) is actually hidden by the real object O_(R), that is, may beestimated. Such estimation may be made on the basis of the number ofother users 11 existing between the user 11-4 and the display object 12,for example, in a direction connecting the user 11-4 and the displayobject 12 (depth direction). In this estimation, it is not necessary toconsider a positional relationship between the users in a direction(horizontal direction) perpendicular to the depth direction when viewedfrom the user 11-4. According to such a configuration, a processing loadapplied to select the first display parameter and the second displayparameter in the technology according to the present disclosure can bereduced. As a result, it is possible to more appropriately change adegree of application of shielding processing.

As a trigger to execute the estimation of the positional relationshipbetween the users, for example, a result of the determination processingwhether or not the other user 11 stays within a predetermined distancefrom the user 11-4 may be adopted. In this determination processing, thepositional relationship between the users is substantially linearlydetermined, instead of planar determination. According to suchdetermination processing, the positional relationship between the usersis estimated at a higher speed, and whether the state is the first stateor the second state can be estimated at a higher speed.

In the first and second embodiments, the first display parameter or thesecond display parameter is selected according to whether the positionalrelationship between the virtual object O_(V) and the real object O_(R)is in the first state or the second state. More specifically, thedisplay method determination unit 104 (or display control unit 105)executes the shielding processing on the virtual object O_(V) on thebasis of the shape of the real object O_(R) according to the firstdisplay parameter in the first state. On the other hand, the displaymethod determination unit 104 (or display control unit 105) executesprocessing of displaying the virtual object O_(V) on the basis of thesecond display parameter while preventing the shielding processing, inthe second state.

That is, the display processing in the technology according to thepresent disclosure may be assumed as processing of determining whetheror not to prevent the shielding processing on the virtual object O_(V).The prevention of the shielding processing may include the inhibition ofthe shielding processing. According to such display processing, aprocessing load required for calculation of the positional relationshipbetween objects for the shielding processing or processing of drawingthe virtual object can be adaptively reduced. The prevention of theshielding processing may include reduction in recognition accuracy ofthe shape of the real object O_(R) for the shielding processing andsimplification of the shape of the recognized real object O_(R). Notethat, regardless of the prevention and the inhibition of the recognitionof the real object O_(R) for the shielding processing, a real space forSimultaneously Localization and Mapping (SLAM) processing related toself-position estimation of the user 11 may be recognized. The shieldingprocessing may be prevented in combination with the estimation of thepositional relationship between the users.

3. Modification

By the way, in the above description, an AR game played by a pluralityof users in cooperation with each other has been described as an exampleof an AR service. However, by using the augmented reality system towhich the technology according to the present disclosure is applied,various AR services can be provided. For example, in the augmentedreality system to which the technology according to the presentdisclosure is applied, an AR navigation service of assisting movement ofa user by the augmented reality (AR) can be provided as an AR service.

In this AR navigation service, for example, when a navigator (virtualobject O_(V)) who is a humanoid navigator guides a user 11 wearing aclient terminal 10 a route (route) to a destination, in a case where thenavigator is hidden by the real object O_(R) such as a corner, abuilding, or the like, display of the virtual object O_(V) and the realobject O_(R) is controlled so as to recognize which real object O_(R)hides the navigator.

(Configuration of Client Terminal And Map Providing Server)

When the augmented reality system to which the technology according tothe present disclosure is applied provides the AR navigation service, amap providing server 30 is provided instead of the position sharingserver 20. FIG. 16 illustrates an example of configurations of theclient terminal 10 and the map providing server 30.

In FIG. 16, the client terminal 10 includes the measurement unit 101 tothe display unit 106 similarly to the client terminal 10 in FIG. 3. Notethat the measurement unit 101 to the display unit 106 in FIG. 16 aredifferent from the measurement unit 101 to the display unit 106 in FIG.3 in content of display method determination processing executed by thedisplay method determination unit 104, and the detailed content thereofwill be described later.

Furthermore, in FIG. 16, the map providing server 30 includes a controlunit 301, a communication unit 302, and a map DB 303.

The control unit 301 includes, for example, a processor such as a CPUand functions as a central processing apparatus that controls anoperation of each unit and executes various arithmetic processing. Thecommunication unit 302 includes a communication module or the likecompliant to wireless communication or wired communication.

The map DB 303 is recorded in a storage device such as a hard disk andmanages three-dimensional map information of various regions as adatabase.

In the map providing server 30 configured as described above, thecommunication unit 302 receives position information transmitted fromthe client terminal 10 via a network 50 according to control from thecontrol unit 301 and transmits map information corresponding to thereceived position information to the client terminal 10 via the network50.

(Example of AR Navigation Service)

FIG. 17 illustrates a scene in which the user 11 receives a guidance ofa route to a destination by a navigator 14 who is a humanoid navigatorby using the AR navigation service.

Here, FIG. 17 is assumed as a field of view when the user 11 wearing theclient terminal 10 views at an outdoor place, and the navigator 14 asthe virtual object O_(V) is displayed in the real world. Note that A ofFIG. 17 illustrates a field of view of the user 11 at a time t21, and Bof FIG. 17 illustrates a field of view of the user 11 at a time t22subsequent to the time t21.

In A of FIG. 17, in the field of view of the user 11, a real object 15such as a road and buildings on the left and right thereof (real objectO_(R) existing on map) and the navigator (virtual object O_(V)) areseen. The navigator 14 guides the user 11 to a direction of thedestination by using a message “Here”.

Thereafter, when it is necessary to pass through a path between thebuildings that are the real objects 15 as the route to the destination,this path is guided by the navigator 14.

At this time, as illustrated in B of FIG. 17, in order to make the user11 recognize which building hides the navigator 14, a target buildingamong the real objects 15 is focused and displayed (for example,highlight). In the example in B of FIG. 17, when the navigator 14 walkson the path between the right building on the front side and theadjacent building among the real objects 15, the front building ishidden and cannot be seen. Therefore, the front building is highlighted(display of dotted pattern in FIG. 17).

Furthermore, here, by changing the display of the navigator 14 shieldedby the front building to the display before being shielded (change solidline in A of FIG. 17 to dotted line in B of FIG. 17), it is possible tomake the user 11 recognize that the navigator 14 exists in the shieldedplace.

Moreover, when the navigator 14 is shielded by a real object 16 in acase where the real object 16 such as a passer (real object O_(R) thatdoes not exist on map) enters the field of view of the user 11, the user11 is made to recognize the state. In the example in B of FIG. 17, threepassers exist in front of the highlighted building and the navigator 14is superimposed and displayed on the passers. Therefore, the user 11 canrecognize that the navigator 14 is shielded by the passers.

In this way, because the client terminal 10 can recognize a relationshipbetween the position of the user 11 and the position of the real object15 such as a building that is the real object O_(R) existing on the mapby using the map information acquired from the map providing server 30,the real object 15 such as a target building can be focused anddisplayed.

Specifically, in a case where real objects O_(R) existing on the mapsuch as buildings 15-1 and 15-2 and real objects O_(R) that do not existon the map such as passers 16-1 to 16-3 exist in the real world asillustrated in FIG. 18, when the virtual object O_(V) is shielded by thereal object O_(R) at the time when the virtual object O_(V) such as thenavigator 14 is presented to the user 11 wearing the client terminal 10,the following display is made.

That is, the client terminal 10 can focus and display the real objectO_(R), (for example, building 15-1) which can be specified by the mapinformation, that shields the virtual object O_(V) (for example,navigator 14). Furthermore, the client terminal 10 can display the realobject O_(R), (for example, passers 16-2 and 16-3) which cannot bespecified by the map information, that shields the virtual object O_(V)(for example, navigator 14) in a state where it is possible to recognizethat the real object O_(R) is shielding the virtual object O_(V).

(Operation of Client Terminal)

Next, an operation of the client terminal 10 will be described withreference to the flowchart in FIG. 19. Note that, here, as in theexamples illustrated in FIGS. 17 and 18, an operation of the clientterminal 10 worn by the user 11 when the AR navigation service isprovided will be described as an example.

In the client terminal 10, a map is loaded (S211). That is, the clientterminal 10 transmits the position information to the map providingserver 30 in advance and acquires three-dimensional map information fromthe map providing server 30 via the network 50 so that a mapcorresponding to a current position (that is, externally generated map)can be loaded.

In step S212, the position information acquisition unit 111 acquires theposition information and the direction information of the user 11.

In step S213, the shielding determination unit 112 determines whether ornot the virtual object O_(V) such as the navigator 14 is hidden by thereal object O_(R) (for example, real object 15 such as building)existing on the map.

When the determination processing in step S213 is affirmative (“YES” inS213), the procedure proceeds to step S214. In step S214, the shieldingrate calculation unit 113 calculates a shielding rate of the real objectO_(R) that is shielding the virtual object O_(V) such as the navigator14 and exists on the map (for example, building).

In step S215, the display parameter setting unit 114 sets a displayparameter that changes display of the virtual object O_(V) to displayaccording to the shielding rate and performs focus display on the realobject O_(R) shielding the virtual object O_(V).

Here, for example, in a case where the navigator 14 is shielded by thereal object 15 which is one or a plurality of buildings or the like atthe time when the AR navigation service is provided, a display parameter(first display parameter) is set that changes the display of a part ofthe navigator 14 shielded by the building to display before shieldingand performs focus display (highlight) on the building that shields thenavigator 14.

When the processing in step S215 ends, the procedure proceeds to stepS216. Furthermore, when the determination processing in step S213 isnegative (“NO” in S213), steps S214 and S215 are skipped, and theprocedure proceeds to step S216.

In step S216, the shielding determination unit 112 determines whether ornot the virtual object O_(V) such as the navigator 14 is hidden by thereal object O_(R) that does not exist on the map (for example, realobject 16 such as passer).

When the determination processing in step S216 is affirmative (“YES” inS216), the procedure proceeds to step S217. In step S217, the shieldingrate calculation unit 113 calculates a shielding rate of the real objectO_(R) that shields the virtual object O_(V) such as the navigator 14 andthat does not exist on the map (for example, passer).

In step S218, the display parameter setting unit 114 sets a displayparameter that changes the display of the virtual object O_(V) accordingto the shielding rate.

Here, for example, in a case where the navigator 14 is shielded by thereal object 16 such as one or a plurality of passers or the like at thetime when the AR navigation service is provided, a display parameter(second display parameter) is set that changes display of a part of thenavigator 14 shielded by the passer to the display before shielding.

When the processing in step S218 ends, the procedure proceeds to stepS219. Furthermore, when the determination processing in step S216 isnegative (“NO” in S216), steps S217 and S218 are skipped, and theprocedure proceeds to step S219.

In step S219, the display control unit 105 controls display of thevirtual object O_(V) and the real object O_(R).

For example, in this display control processing, when the determinationprocessing in step S213 is affirmative (“YES” in S213), the display ofthe virtual object O_(V) and the real object OR is controlled on thebasis of the display parameter set in the processing in step S215. Morespecifically, as illustrated in B of FIG. 17, display of the navigator14 shielded by the front building is changed from the display beforeshielding, and the front building is focused and displayed (highlight).

Furthermore, for example, in the display control processing, when thedetermination processing in step S216 is affirmative (“YES” in S216),the display of the virtual object O_(V) is controlled on the basis ofthe display parameter set in the processing in step S218. Morespecifically, as illustrated in B of FIG. 17, the display of thenavigator 14 shielded by the passer on the front side is changed fromthe display before shielding.

When the processing in step S219 ends, the procedure proceeds to stepS220. In step S220, it is determined whether or not to end theprocessing.

When the determination processing in step S220 is negative (“NO” inS220), the procedure returns to step S212, and the processing in stepsS212 to S220 is repeated, and then, the display of the virtual objectO_(V) and the real object O_(R) is controlled. Furthermore, when thedetermination processing in step S220 is affirmative (“YES” in S220),the processing illustrated in FIG. 19 ends.

The operation of the client terminal 10 has been described above.

Note that, in the above description, as the augmented reality system inFIG. 1, a configuration is indicated in which the client terminals 10-1to 10-N share the position information and the direction information byexchanging the data with the position sharing server 20 via the network50. However, the client terminals 10-1 to 10-N may share the positioninformation and the direction information by each directly exchangingdata without providing the position sharing server 20.

Furthermore, in the above description, the position information and thedirection information are exemplified as the information shared by theclient terminals 10-1 to 10-N. However, it is not necessarily to sharethe direction information, and in addition, information other than theposition information and the direction information may be shared.Moreover, as the position information shared by the client terminals10-1 to 10-N, for example, an absolute position based on a signalobtained by the Global Positioning System (GPS) or the like may be used,in addition to, for example, a relative position with respect to thereal object O_(R), the virtual object O_(V), or the like.

Furthermore, the client terminal 10 is configured as an electronicapparatus such as a wearable terminal, for example, a glass-typeinformation terminal, a head mounted display (HMD), or the like.However, the display unit 106 of the client terminal 10 may be, forexample, a transmissive display that displays a video on the inner sideof the lens of the glasses or may be a closed type display that(completely) covers the field of view of the user 11.

Moreover, in the above description, each client terminal 10 generatesthe display parameter. However, by transmitting information (forexample, information such as position information, directioninformation, or the like) used to generate the display parameter to aserver on the network 50, and the server may generate (intensivelyprocess) the display parameter used by each client terminal 10 andtransmit the display parameter to each client terminal 10 via thenetwork 50.

Note that, in the network 50, not only wired communication and wirelesscommunication, but also communication in which wireless communicationand wired communication are mixed may be performed, that is, wirelesscommunication may be performed in a certain section, and wiredcommunication may be performed in another section. Moreover, in thenetwork 50, communication from a certain device to another device may beperformed by wired communication, and communication from the otherdevice to the certain device may be performed by wireless communication.

4. Configuration of Computer

The series of processing described above (for example, operation ofclient terminal 10 illustrated in FIG. 9 or 19) can be executed byhardware or software. When the series of the processing is performed bythe software, a program included in the software is installed in acomputer of each device. FIG. 20 is a block diagram illustrating anexemplary configuration of hardware of the computer for executing theabove-mentioned series of processing by the program.

In a computer 1000, a Central Processing Unit (CPU) 1001, a Read OnlyMemory (ROM) 1002, and a Random Access Memory (RAM) 1003 are connectedto each other with a bus 1004. In addition, an input/output interface1005 is connected to the bus 1004. An input unit 1006, an output unit1007, a recording unit 1008, a communication unit 1009, and a drive 1010are connected to the input/output interface 1005.

The input unit 1006 includes a microphone, a keyboard, a mouse, and thelike. The output unit 1007 includes a speaker, a display, and the like.The recording unit 1008 includes a hard disk, a non-volatile memory, andthe like. The communication unit 1009 includes a network interface andthe like. The drive 1010 drives a removable recording medium 1011 suchas a magnetic disk, an optical disk, an optical magnetic disk, or asemiconductor memory.

In the computer 1000 configured as above, the CPU 1001 executes theprogram recorded in the ROM 1002 and the recording unit 1008 by loadingit to the RAM 1003 via the input/output interface 1005 and the bus 1004.According to this, the above-mentioned series of processing is executed.

The program executed by the computer 1000 (CPU 1001) can be provided,for example, by recording it to the removable recording medium 1011 as apackage medium and the like. Furthermore, the program can be providedthrough wireless or wired transmission media such as a local areanetwork, the Internet, and a digital satellite broadcast.

In the computer 1000, the program can be installed to the recording unit1008 via the input/output interface 1005 by mounting the removablerecording medium 1011 in the drive 1010. Furthermore, the program can bereceived by the communication unit 1009 via the wired or wirelesstransmission media and can be installed to the recording unit 1008. Inaddition, the program can be previously installed to the ROM 1002 andthe recording unit 1008.

Here, in the present specification, it is not necessary to execute theprocessing, which is executed by the computer according to the program,in an order described in the flowchart in time series. That is, theprocessing executed by the computer according to the program includesprocessing executed in parallel or individually (for example, parallelprocessing or processing by object). Furthermore, the program may beperformed by a single computer (processor), and distributed processingof the program may be executed by a plurality of computers.

Note that the embodiment of the present technology is not limited to theabove-mentioned embodiments, and various changes can be made withoutdeparting from the scope of the technology according to the presentdisclosure.

Furthermore, each step of the series of processing (for example,operation of client terminal 10 illustrated in FIG. 9 or 19) can beexecuted by a single device or can be shared and executed by a pluralityof devices. Moreover, when a plurality of kinds of processing isincluded in one step, the plurality of kinds of processing included inone step can be executed by the single device or can be divided andexecuted by the plurality of devices.

Note that the technology according to the present disclosure can havethe following configurations.

(1)

An information processing apparatus including:

a display control unit configured to control a display so as to

-   -   display a virtual object by using a first display parameter in a        first state where it is determined that the virtual object        displayed by the display is hidden by at least one real object        as viewed from a first user of the display and    -   display the virtual object by using a second display parameter        different from the first display parameter in a second state        where it is determined that the virtual object is hidden by real        objects more than that in the first state as viewed from the        first user.

(2)

The information processing apparatus according to (1), furtherincluding:

a determination unit configured to determine a degree of shielding ofthe virtual object by the real object, in which

the display control unit controls display of the virtual object by usinga display parameter corresponding to a state according to adetermination result of the degree of shielding by the determinationunit.

(3)

The information processing apparatus according to (1) or (2), in which

each of the first display parameter and the second display parameter isassociated with at least a single attribute of a display position, adisplay posture, a size, a color, or a transmittance with respect to thereal object of the virtual object.

(4)

The information processing apparatus according to (3), in which

the first display parameter is a parameter used to display the virtualobject at a first position according to the degree of shielding, and

the second display parameter is a parameter used to display the virtualobject at a second position different from the first position accordingto the degree of shielding.

(5)

The information processing apparatus according to (4), in which

the first display parameter is a parameter used to change the displayposition of the virtual object to the first position, and

the second display parameter is a parameter used to change the displayposition of the virtual object to the second position.

(6)

The information processing apparatus according to (4) or (5), in which

at least one of the first display parameter or the second displayparameter is a parameter to move the virtual object from a defaultdisplay position to an upper position.

(7)

The information processing apparatus according to any one of (4) to (6),in which

the virtual object is an object that is not fixed to a surface in a realspace.

(8)

The information processing apparatus according to (3), in which

the first display parameter is a parameter used to display the virtualobject at a first transmittance according to the degree of shielding,and

the second display parameter is a parameter used to display the virtualobject at a second transmittance different from the first transmittanceaccording to the degree of shielding.

(9)

The information processing apparatus according to (8), in which

the first transmittance and the second transmittance are higher thanzero % and are lower than 100% when it is assumed that completetransmission be 100% and complete opacity be zero %.

(10)

The information processing apparatus according to (9), in which

the first transmittance is a value lower than the second transmittance.

(11)

The information processing apparatus according to any one of (1) to(10), in which

the real object is a dynamic object.

(12)

The information processing apparatus according to (11), in which

the dynamic object includes a person.

(13)

The information processing apparatus according to (12), in which

the person is a second user who is different from the first user andreceives provision of a service same as the first user receives.

(14)

The information processing apparatus according to (13), furtherincluding:

an acquisition unit configured to acquire position information of thefirst user and position information of the second user; and

a setting unit configured to set the first display parameter and thesecond display parameter on the basis of the position information of thefirst user and the position information of the second user.

(15)

The information processing apparatus according to (13) or (14), in which

the service is a service using Augmented Reality (AR).

(16)

The information processing apparatus according to any one of (1) to(15), in which

the display control unit

-   -   executes shielding processing on the virtual object on the basis        of a shape of the real object on the basis of the first display        parameter in the first state, and    -   executes display processing on the virtual object on the basis        of the second display parameter while preventing the shielding        processing, in the second state.

(17)

The information processing apparatus according to any one of (1) to(16), configured as a mobile terminal including the display.

(18)

The information processing apparatus according to (17), in which

the mobile terminal includes a wearable terminal or a head mounteddisplay.

(19)

An information processing method including:

controlling a display by an information processing apparatus so as to

-   -   display a virtual object by using a first display parameter in a        first state where it is determined that the virtual object        displayed by the display is hidden by at least one real object        as viewed from a first user of the display and    -   display the virtual object by using a second display parameter        different from the first display parameter in a second state        where it is determined that the virtual object is hidden by real        objects more than that in the first state as viewed from the        first user.

(20)

A program for causing a computer to function as

a display control unit that controls a display so as to

-   -   display a virtual object by using a first display parameter in a        first state where it is determined that the virtual object        displayed by the display is hidden by at least one real object        as viewed from a first user of the display and    -   display the virtual object by using a second display parameter        different from the first display parameter in a second state        where it is determined that the virtual object is hidden by real        objects more than that in the first state as viewed from the        first user.

REFERENCE SIGNS LIST

-   10, 10-1 to 10-N Client terminal-   20 Position sharing server-   30 Map providing server-   50 Network-   101 Measurement unit-   102 Position estimation unit-   103 Communication unit-   104 Display method determination unit-   105 Display control unit-   106 Display unit-   111 Position information acquisition unit-   112 Shielding determination unit-   113 Shielding rate calculation unit-   114 Display parameter setting unit-   115 Attribute determination unit-   201 Control unit-   202 Communication unit-   301 Control unit-   302 Communication unit-   303 Map database-   1000 Computer-   1001 CPU-   O_(R) Real object-   O_(V) Virtual object

1. An information processing apparatus comprising: a display controlunit configured to control a display so as to display a virtual objectby using a first display parameter in a first state where it isdetermined that the virtual object displayed by the display is hidden byat least one real object as viewed from a first user of the display anddisplay the virtual object by using a second display parameter differentfrom the first display parameter in a second state where it isdetermined that the virtual object is hidden by real objects more thanthat in the first state as viewed from the first user.
 2. Theinformation processing apparatus according to claim 1, furthercomprising: a determination unit configured to determine a degree ofshielding of the virtual object by the real object, wherein the displaycontrol unit controls display of the virtual object by using a displayparameter corresponding to a state according to a determination resultof the degree of shielding by the determination unit.
 3. The informationprocessing apparatus according to claim 2, wherein each of the firstdisplay parameter and the second display parameter is associated with atleast a single attribute of a display position, a display posture, asize, a color, or a transmittance with respect to the real object of thevirtual object.
 4. The information processing apparatus according toclaim 3, wherein the first display parameter is a parameter used todisplay the virtual object at a first position according to the degreeof shielding, and the second display parameter is a parameter used todisplay the virtual object at a second position different from the firstposition according to the degree of shielding.
 5. The informationprocessing apparatus according to claim 4, wherein the first displayparameter is a parameter used to change the display position of thevirtual object to the first position, and the second display parameteris a parameter used to change the display position of the virtual objectto the second position.
 6. The information processing apparatusaccording to claim 5, wherein at least one of the first displayparameter or the second display parameter is a parameter to move thevirtual object from a default display position to an upper position. 7.The information processing apparatus according to claim 6, wherein thevirtual object is an object that is not fixed to a surface in a realspace.
 8. The information processing apparatus according to claim 3,wherein the first display parameter is a parameter used to display thevirtual object at a first transmittance according to the degree ofshielding, and the second display parameter is a parameter used todisplay the virtual object at a second transmittance different from thefirst transmittance according to the degree of shielding.
 9. Theinformation processing apparatus according to claim 8, wherein the firsttransmittance and the second transmittance are higher than zero % andare lower than 100% when it is assumed that complete transmission be100% and complete opacity be zero %.
 10. The information processingapparatus according to claim 9, wherein the first transmittance is avalue lower than the second transmittance.
 11. The informationprocessing apparatus according to claim 1, wherein the real object is adynamic object.
 12. The information processing apparatus according toclaim 11, wherein the dynamic object includes a person.
 13. Theinformation processing apparatus according to claim 12, wherein theperson is a second user who is different from the first user andreceives provision of a service same as the first user receives.
 14. Theinformation processing apparatus according to claim 13, furthercomprising: an acquisition unit configured to acquire positioninformation of the first user and position information of the seconduser; and a setting unit configured to set the first display parameterand the second display parameter on a basis of the position informationof the first user and the position information of the second user. 15.The information processing apparatus according to claim 13, wherein theservice is a service using Augmented Reality (AR).
 16. The informationprocessing apparatus according to claim 1, wherein the display controlunit executes shielding processing on the virtual object on a basis of ashape of the real object on a basis of the first display parameter inthe first state, and executes display processing on the virtual objecton a basis of the second display parameter while preventing theshielding processing, in the second state.
 17. The informationprocessing apparatus according to claim 1, configured as a mobileterminal including the display.
 18. The information processing apparatusaccording to claim 17, wherein the mobile terminal includes a wearableterminal or a head mounted display.
 19. An information processing methodcomprising: controlling a display by an information processing apparatusso as to display a virtual object by using a first display parameter ina first state where it is determined that the virtual object displayedby the display is hidden by at least one real object as viewed from afirst user of the display and display the virtual object by using asecond display parameter different from the first display parameter in asecond state where it is determined that the virtual object is hidden byreal objects more than that in the first state as viewed from the firstuser.
 20. A program for causing a computer to function as a displaycontrol unit that controls a display so as to display a virtual objectby using a first display parameter in a first state where it isdetermined that the virtual object displayed by the display is hidden byat least one real object as viewed from a first user of the display anddisplay the virtual object by using a second display parameter differentfrom the first display parameter in a second state where it isdetermined that the virtual object is hidden by real objects more thanthat in the first state as viewed from the first user.